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ABSTRACT 


Military  recruiting  and  advertising  expenditures  by  the  United  States  Navy  totaled  $42.6 
million  in  1995.  Commander,  Navy  Recruiting  Command  (CNRC)  is  responsible  for  executing 
the  Navy's  advertising  budget.  The  problems  faced  by  CNRC  lie  in  determining  if  the  FY  1994 
and  FY  1995  increases  in  advertising  dollars  fulfilled  their  objective  of  increasing  national 
awareness.  This  study  examines  the  increases  in  Navy  national  general  enlisted  advertising 
budgets  to  determine  if  additional  spending  resulted  in  (1)  an  increase  in  national  awareness 
among  the  target  population,  as  measured  by  the  number  of  leads,  or  inquiries,  generated,  and 
(2)  an  increase  in  the  influence  of  advertising  on  new  recruits,  as  measured  by  the  New  Recruit 
Survey  (NRS)  responses  during  the  period  July  1994  through  August  1995.  To  provide  an 
analysis  of  the  effects  of  advertising  expenditures,  this  thesis  investigates  national  advertising 
data  by  media  type  (television,  radio,  magazine,  and  direct  mail),  and  the  number  of  leads 
generated  at  the  Navy  Recruiting  District  (NRD)  level.  Due  to  data  limitations,  findings  of  this 
study  are  limited.  Correlations  between  advertising  expenditures  and  leads  generated  at  the 
recruiting  district  level  revealed  little  interdependence;  however,  a  four  month  advertising  lag 
effect  emerged.  Trend  analysis  of  expenditures  versus  leads  revealed  no  real  change  due  to  the 
increased  budget.  Lead  patterns  remain  constant  despite  additional  advertising.  However, 
evaluation  of  the  New  Recruit  Surveys  revealed  that  national  advertising  creates  awareness  and 
has  a  positive  influence  on  an  individual's  decision  to  join  the  Navy. 
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I.  INTRODUCTION 


A.  BACKGROUND 

Advertising  has  long  been  used  by  all  branches  of  the  United  States 
Armed  Forces.  Posters  and  handbills  provide  some  of  the  earliest  evidence  of 
efforts  to  use  advertising  as  a  means  of  recruiting.  However,  the  magnitude  of 
advertising  has  varied  throughout  history.  The  Navy  spent  little  more  than  a 
million  dollars  on  all  of  its  advertising  programs  in  1964,  less  than  .01%  of  all 
Navy  expenditures  in  FY  1964  (Arima,  1978).  In  fact,  in  FY  1965,  in  the  midst  of 
the  Vietnam  conflict,  the  Navy's  advertising  budget  dipped  below  the  million 
dollar  mark. 

The  introduction  of  the  All  Volunteer  Force  (AVF)  in  July  1973  changed 
the  role  of  Navy  advertising  as  it  presented  a  major  new  marketing  challenge. 
There  no  longer  existed  a  "guaranteed"  manpower  level  of  qualified  males 
whose  only  choice  was  among  which  branch  of  the  military  to  serve.  The  Navy 
was  faced  with  the  task  of  competing  not  only  against  the  other  armed  services, 
but  with  the  civilian  labor  market  as  well.  This  required  the  development  of  a 
new  marketing  strategy  in  which  enlistments  were  transacted  in  large  quantities 
to  adequately  staff  the  naval  forces  in  numbers  as  well  as  in  required  quality. 

The  reality  of  the  AVF  resulted  in  advertising  becoming  a  major  portion  of 
the  Navy's  new  recruiting  mission.  Advertising  expenditures  quadrupled,  in  real 
terms,  between  FY  1971  and  FY  1972  and  tripled  in  FY  1973,  the  year  American 
involvement  in  the  Vietnam  conflict  concluded  and  the  draft  ended  (Arima, 
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1 978).  The  first  year  of  the  AVF,  FY  1 974,  was  a  peak  year  for  advertising 
expenditures;  the  Navy  spent  $26.8M  (then  year  dollars).  Spending  slowly 
decreased  as  the  AVF  took  hold,  plateauing  at  about  six  times  the  level  of  the 
last  year  completely  unaffected  by  the  AVF,  FY  1971. 

The  end  of  the  Cold  War  plunged  the  military  into  an  era  dictated  by 
significantly  different  circumstances  than  those  encountered  in  previous 
decades.  The  imposing  threat  to  national  security  was  abruptly  ended  and  the 
armed  forces  found  themselves  floundering  to  gain  a  foothold  in  the  realm  of  the 
unknown  future.  Some  of  the  major  impacts  of  these  changes  have  been  the 
downsizing  of  the  Navy's  end-strength  targets,  reduced  budgets  and  diminishing 
advertising  funds.  As  indicated  by  Webb  and  Sohn  (1994),  albeit  forces  have 
been  cut  across  all  ranks,  it  was  the  public's  perception  that  a  smaller  Navy  was 
achieved  by  means  of  curtailed  enlistments.  This  perception  resulted  in  the 
Navy's  advertising  budget  being  reduced  from  approximately  $42  million  in  FY 
1986  to  only  $15.1  million  in  FY  1992.  A  record  of  advertising  expenditures  by 
the  Navy  over  the  last  sixteen  years  is  presented  in  Table  1  and  illustrated 
graphically  in  Figure  1 . 

Contrary  to  public  perception,  mandated  enlistment  numbers  remain 
relatively  constant.  This  has  been  necessary  to  compensate  for  attrition  among 
enlistees  through  their  first  enlistment.  In  an  effort  to  increase  public  awareness 
and  to  compete  for  high  quality  enlistees,  Congress  virtually  doubled  the  Navy's 
advertising  budget  in  FY  1994  to  $30.2  million.  The  purpose  of  this  research 
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Table  1 

Annual  Navy  Advertising  Expenditures 
(in  millions  of  dollars) 


FY 

Current  Dollars^ 

Constant  D( 

80 

19.0 

32.2 

81 

18.9 

29.0 

82 

16.3 

23.9 

83 

15.5 

22.3 

84 

13.1 

18.4 

85 

20.0 

27.4 

86 

31.0 

42.0 

87 

14.9 

19.6 

88 

19.4 

24.6 

89 

19.3 

23.6 

90 

25.1 

29.5 

91 

16.6 

18.5 

92 

13.9 

15.1 

93 

15.5 

16.3 

94^ 

29.7 

30.2 

95 

36.0 

36.0 

96 

37.0 

36.0 

^  Source:  Navy  Recruiting  Command. 

^  Calendar  Year  1 995  =  1 00 

^  FY94-96  figures  do  not  include  postage.  Prior  to  FY94,  individual  commands  were  not  required 
to  pay  their  own  postage. 
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Fiscal  Year 

Figure  1.  Annual  Advertising  Expenditures 
FY80-FY96  in  Constant  1995  Doilars 


effort  is  to  analyze  the  cost  effectiveness  of  the  Navy's  general  enlisted  national 
advertising  program. 

Commander,  Navy  Recruiting  Command  (CNRC)  is  responsible  for 
executing  the  Navy's  advertising  budget.  The  problems  faced  by  CNRC  lie  in 
determining  if  the  increase  in  advertising  dollars  has  increased  national 
awareness.  The  measure  used  in  this  study  to  determine  awareness  is  the 
number  of  leads  generated,  or  the  number  of  inquiries  generated  by  national 
advertising.  While  the  number  of  leads  generated  do  not  encompass  all 
possible  awareness  factors  (i.e.,  parents,  teachers,  friends),  it  does  permit  a 
means  of  measuring  awareness  among  the  target  audience.  The  bridge 
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between  awareness  and  sales  is  rarely  crossed  by  national  advertising 
agencies.  However,  in  an  environment  which  looks  to  justify  smaller  budgets, 
the  link  between  the  two  must  be  ascertained  by  the  Navy  to  determine  the 
effectiveness  of  advertising  dollars  being  spent. 

B.  OBJECTIVE 

The  objective  of  this  thesis  is  to  provide  an  analytical  evaluation  of  the 
affects  of  changes  in  budgetary  dollars  for  Navy  national  advertising  from  FY 
1993  through  second  quarter  FY  1995.  Given  the  changes  (plus-ups)  in  the 
budget,  this  study  determines  if  there  is  any  evidence  that  the  increase  in 
advertising  has  caused  an  increase  in  national  awareness,  as  measured  by  the 
number  of  leads  generated.  In  addition,  it  determines  if  there  has  been  an 
increase  in  the  influence  of  advertising  on  new  recruits,  as  measured  by  the  New 
Recruit  Survey  results  from  July  1994  through  April  1995. 

C.  RESEARCH  QUESTIONS 

The  following  specific  questions  are  addressed  in  this  research: 

1.  Has  the  increase  in  the  Navy  advertising  budget  had  an  impact  on 
national  advertising  awareness  as  measured  by  the  number  of  leads  generated? 

2.  How  much  of  the  budget  is  spent  on  advertising  execution  and  how 
much  is  considered  production  overhead? 

3.  What  is  the  relationship  between  changes  in  the  Navy  advertising 
budget  and  changes  in  the  number  of  leads  generated? 
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4.  What  is  the  relationship  between  the  number  of  impressions'*  media 
dollars  buy  and  the  number  of  leads  generated? 

5.  Given  successful  execution,  is  there  evidence  that  Navy  advertising 
increases  national  awareness? 

6.  Is  there  evidence  that  the  increase  in  the  Navy  advertising  budget 
affects  attitudes  of  new  recruits? 

7.  Does  the  evidence  suggest  some  media  types  are  more  cost-effective 
in  terms  of  the  number  of  leads  they  produce? 

D.  LITERATURE  REVIEW 

Numerous  studies  since  the  conclusion  of  the  draft  analyzed  the  effects  of 
economic  factors,  advertising  expenditures,  and  various  recruiting  assets  on 
enlistments.  The  consensus  among  studies  conducted  for  the  Navy  was  that 
advertising's  impact  on  the  supply  of  recruits  was  relatively  weak  in  comparison 
to  environmental  factors,  economic  factors,  and  recruiter  motivation.  (Bayus,  et 
al.  1985,  Goldberg,  1982,  Hanssens  and  Levien,  1983,  and  Morey  and  McCann, 
1980)  This  conclusion  was  based  on  a  failure  to  establish  statistical 
relationships  between  the  varying  levels  of  advertising  expenditures  and 
corresponding  number  of  enlistments  (Dertouzos,  1989). 

The  majority  of  advertising  agencies  and  advertising  research  firms  that 
perform  research  on  the  effectiveness  of  advertising  state  that  the  important 
factors  to  measure  are  awareness  of  and  attitude  toward  the  product  or  service 


An  impression  is  defined  as  the  exposure  of  one  individual  to  one  advertisement.  Impressions 
will  be  further  discussed  in  Chapter  III. 


6 


being  advertised.  Rather  than  budget  levels,  their  concern  is  with  the  message, 
the  segmentation  of  the  population  into  target  audiences,  and  the  reach  and 
frequency  with  which  audiences  are  contacted.  The  bridge  between  awareness 
and  attitudes  on  one  hand,  and  leads  or  recruits,  on  the  other,  is  rarely  crossed. 

A  number  of  models  have  been  developed  which  deal  with  the  various 
features  of  military  enlistment  supply  and  demand.  RAND  researchers 
developed  an  early  model  which  captured  the  effects  of  advertising.  Using  Army 
advertising  expenditure  data  from  the  early  1980s,  the  study  gauged  the 
effectiveness  of  Army  advertising  over  a  36-month  period,  FY  1 982  through  FY 
1984.  The  empirical  results  showed  that,  in  general,  advertising  expenditures  in 
a  given  month  had  a  significant  and  immediate  effect  on  the  number  of  high- 
quality  Army  enlistments.  In  addition,  the  study  suggested  that  at  the  budget 
and  operational  levels  which  prevailed  in  the  1980s,  Army  advertising 
expenditures  were  made  in  a  cost-effective  manner.  (Dertouzos  and  Polich, 
1989) 

The  Dertouzos  and  Polich  (1989)  study  accumulated  geographical  data 
for  advertising,  recruiting  goals,  and  goal  attainment  both  at  the  national  and 
local  levels.  Significant  variation  was  noted  in  both  the  advertising  mix  and 
expenditure  levels  for  local  and  national  advertising.  The  variation  was 
determined  to  exist  both  over  time  and  across  geographical  location.  "...  after 
accounting  for  variations  across  areas,  [the]  analysis  suggests  that  there  is  a 
one-to-one  correspondence  between  changes  in  expenditures  and  changes  in 
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impressions."  (Dertouzos  and  Polich,  1989,  pi 7).  In  addition, "...  once 
systematic  differences  across  [areas]  are  taken  into  account,  cost  data 
accurately  reflect  the  penetration  of  advertising."  (Dertouzos  and  Polich,  1989, 
pi  7).  Cost  data  was  subsequently  used  in  lieu  of  audience  or  rating  data  to 
represent  advertising  in  specific  media. 

Bayus  et  al.  (1985)  examined  the  market  effectiveness  of  the  Navy's 
recruiting  program  and  attempted  to  quantify  the  relationship  between  marketing 
efforts  and  enlistments.  They  performed  a  one-year  controlled  experiment  in 
which  numbers  of  Navy  recruiters  and  levels  of  advertising  were  systematically 
varied.  Monthly  data  for  FY  1980  was  gathered  into  a  comprehensive  data  base 
and  evaluated  by  quality  of  contracts. 

Bayus  et  al.  found  recruiters  to  have  the  most  constant  effect  on 
enlistments  across  all  contract  types.  It  was  also  discovered  there  were 
"...  variations  of  recruiter  effectiveness  based  on  both  calendar  period  and 
recruiter  tenure"  (p372).  Navy  national  advertising  did  not  have  any  significant 
effect;  however,  the  effect  of  local  advertising  was  strong.  The  study  attributed 
this  effect  of  local  advertising  to  the  possibility  that "...  local  advertising 
stimulates  local  leads  and  walk-in  traffic  in  lieu  of  national  leads".  (p371 ) 

Finally,  socioeconomic  factors  such  as  unemployment  and  urbanization  were 
found  to  have  major  impacts  on  the  enlistment  numbers  and  quality  of  enlistees. 

Hanssens  and  Levien  (1983)  utilized  historical  data  at  the  district  level 
from  1976-1978.  Total  Navy  contracts  were  analyzed  by  month  and  recruiting 
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district.  An  econometric  model  quantified  the  effects  of  various  environmental 
and  marketing  variables  on  nationally  generated  leads  and  contracts. 

The  econometric  model  revealed  that  both  environmental  and  marketing 
variables  had  significant  impacts  on  recruiting  performance  over  time  and  across 
recruiting  districts.  National  advertising  was  positively  correlated  with  national 
leads;  however,  there  was  no  real  effect  on  contracts  signed.  In  addition,  local 
advertising  negatively  influenced  national  leads.  It  was  suggested  that  there 
was  no  real  lag  effect  for  recruiters  on  leads  or  contracts  signed,  and  only  a  one 
month  lag  effect  for  advertising. 

Warner  (1990)  examined  trends  in  military  recruiting  during  the  1980s. 

The  findings  of  the  study  were  consistent  with  the  many  previous  studies  of  Army 
enlistment  supply.  Changing  the  number  of  recruiters  and  the  advertising 
budget  were  determined  to  be  the  most  cost-effective  means  of  varying  high- 
quality  enlistments.  More  interestingly,  the  study  also  concluded  that 
cooperation,  rather  than  competition,  existed  among  the  military  recruiting 
programs.  In  particular,  expanding  the  recruiter  forces  of  one  service  seemed  to 
increase  the  enlistments  of  all  services. 

In  summary,  the  academic  literature  on  military  recruiting  has  been 
focused  on  socioeconomic  effects  and  recruiter  efforts.  In  analyzing  the 
effectiveness  of  advertising,  there  does  not  appear  to  be  many  attempts  linking 
the  awareness  advertising  creates  to  any  measurable  outcome,  such  as  leads 
generated  or  number  of  recruits. 


II.  NAVY  ADVERTISING  PROGRAM 


A.  ORGANIZATION 

CNRC,  headquartered  in  Arlington,  Virginia,  is  responsible  for  carrying 
out  the  Navy's  recruiting  effort.  In  1995,  CNRC  was  responsible  for  recruiting 
more  than  48,000  individuals,  an  accomplishment  that  used  a  budget  of  42.25 
million  dollars  and  a  force  of  more  than  4000  recruiters  and  support  staff.  These 
recruiters  are  distributed  throughout  the  geographic  United  States,  which  is 
divided  into  four  regional  areas  or  Navy  Recruiting  Areas  (NRAs).  The  NRAs 
are  further  divided  into  31  Navy  Recruiting  Districts  (NRDs)  as  listed  in  Table  2. 
The  geographic  orientation  of  the  NRAs  and  NRDs  is  illustrated  in  Figure  2. 
Districts  are  split  into  185  zones,  and  the  zones  into  1176  stations.  Stations 
provide  face  to  face  sales  for  Navy  recruiting.  The  recruiting  effort  is  divided  into 
several  marketing  medium  including:  television,  direct  mail,  radio,  magazine, 
paraphernalia,  and  personal  contact. 

Two  additional  field  activities  operate  under  CNRC.  The  Navy  National 
Recruiting  Information  Center  (NNRIC)  in  Rockville,  Maryland,  receives  toll-free 
telephone  inquiries  to  1-800-USA-NAVY  in  response  to  Navy  ads.  The  inquiries 
are  forwarded  to  the  second  activity.  Navy  Opportunity  and  Information  Center 
(NOIC)  in  Gaithersburg,  Maryland.  Both  NNRIC  and  NOIC  serve  as  collection 
centers  for  leads  generation  data  and  are  not  sources  for  leads.  The  mission  of 
NOIC  is  to  intake,  process,  and  respond  to  inquiries  from  the  public 
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Table  2 

1 995  Navy  Recruiting  Areas  and  Districts 


NRA  Code  Name 
01  Scotia,  NY 


03  Macon,  GA 


05  Great  Lakes,  IL 


08  San  Francisco,  CA 


NRD  Code 

Name 

02 

New  England 

03 

Buffalo 

04 

New  York 

18 

Columbus 

19 

Philadelphia 

20 

Pittsburgh 

22 

Michigan 

10 

Montgomery 

12 

Jacksonville 

13 

Atlanta 

14 

Cumberland  Valley 

15 

Raleigh 

16 

Richmond 

34 

New  Orleans 

48 

Miami 

21 

Chicago 

27 

Kansas  City 

28 

Minneapolis 

29 

Omaha 

31 

Dallas 

32 

Houston 

42 

Indianapolis 

47 

St.  Louis 

25 

Denver 

30 

Albuquerque 

36 

Los  Angeles 

37 

Portland 

38 

San  Francisco 

39 

Seattle 

40 

San  Diego 

46 

San  Antonio 

12 
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about  opportunities  in  the  US  Navy.  The  inquiries  may  be  received  from 
business  reply  cards,  unsolicited  inquiries  (termed  "white  mail"),  or  calls  to  the  1- 
800  telephone  number.  Once  the  inquiries  are  made,  they  are  evaluated  as 
eligible  or  not  eligible.  Eligibility  is  determined  based  on  the  following  criteria: 

Age:  16.5  to  34.9 

Education:  1 1  years  of  school  completed  during  July-April,  inclusive,  or 
at  least  10  years  completed  during  May-June 
The  day  following  an  inquiry,  fulfillment  of  eligible  leads  is  carried  out  by  sending 
the  inquirer  an  "EG"  (general  enlisted)  computer  letter,  "Today's  Navy"  booklet, 
and  a  College  Fund  Fact  Sheet.  At  the  same  time,  a  postcard  is  sent  to  the 
corresponding  recruiter  in  the  same  district  with  information  pertaining  to  the 
lead  for  follow-up  action. 

The  Marketing  Communications  Department  (Code  80)  at  CNRC  is 
divided  into  five  divisions,  representing  the  functional  breakdown  of  the 
advertising  effort: 

Code  81  Public  Affairs  Division 

Code  82  Director  of  Education  and  Youth  Programs  Division 

Code  83  Creative  Services  Division 

Code  84  Management  Services  Division 

Code  85  Advertising  Operations  Division 

The  national  advertising  effort  works  through  all  of  these  divisions  to  create  and 
market  programs  on  a  national  level.  The  advertising  agency  Batten,  Barton, 


Durstein  and  Osborne  (BBDO)  handles  the  majority  of  advertising  for  national 
programs  and  media  (television,  radio,  and  magazines).  There  is  a  separate 
agency  which  handles  direct-mail  services,  Rapp  Collins  Worldwide  (RCW). 

Recruiting  aids  (called  "RADs")  are  planned,  created,  and  distributed 
through  in-house  efforts  by  the  Creative  Services  Division.  The  promotion  of 
public  service  advertising  is  also  handled  in-house.  Finally,  the  Management 
Services  Division  creates  the  communications  plans  and  tracks  and  monitors 
overall  program  progress.  One  of  the  tracking  systems  used  is  the  Navy 
Advertisement  Lead  Tracking  System  (NALTS)  which  tracks  the  distribution  of 
leads  received  through  the  toll-free  telephone  number.  Leads  are  tracked  in 
numerous  ways,  for  example,  by  NRA,  NRD,  advertising  program,  or  agency. 
The  NALTS  system  is  continuously  updated  by  recruiters  and  districts  as  leads 
information  changes. 

B.  PROGRAM  AREAS 

Navy  advertising  programs  are  closely  aligned  with  the  recruiting 
programs  they  support.  A  majority  of  the  advertising  budget  can  be  directly 
linked  with  these  recruiting  programs.  Recruiting  is  initially  broken  down  into 
enlisted  and  officer  categories;  there  are  general  enlisted  and  general  officer 
programs.  Both  of  the  general  programs  also  have  programs  specifically  geared 
toward  minority  recruiting.  The  enlisted  program  is  further  separated  into 
categories  for  reserves  and  occupational  specialty.  The  officer  program  is 
segregated  by  occupational  specialty,  such  as  medical  or  chaplain  programs,  or 
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source  of  commissioning,  such  as  NROTC.  The  general  enlisted  national 
advertising  program  is  the  focus  of  this  study. 

The  allocation  of  national  advertising  funds  to  major  program  areas  is 
shown  in  Figure  3,  based  on  the  FY  1995  Close-out  Report.  Almost  three- 
fourths  of  national  advertising  funds  go  to  the  general  enlisted  program  alone. 
The  remainder  goes  to  officer  programs  and  postage.  The  ratio  of  enlisted  to 
officer  advertising  is  4-to-1.  Of  note,  until  FY  1994,  individual  commands  did  not 
pay  their  own  postage;  since  then,  postage  has  consumed  a  good  portion  of  the 
advertising  budget. 


Figure  3.  Allocation  of  Navy  National  Advertising  Budget 
to  Major  Program  Areas  in  FY  1995 
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C.  GENERAL  ENLISTED  ADVERTISING  PROGRAM 


The  general  enlisted  program  is  an  umbrella  program  whose  target  is  18- 
to-24  year  olds  with  an  emphasis  on  18-to-21  year  olds  who  have  graduated 
from  high  school,  are  in  the  upper  half  of  this  group  in  general  mental  aptitude, 
and  are  mechanically  and  technically  oriented.  The  basis  to  this  approach  is  a 
unique  selling  proposition  (DSP)  that  distinguishes  the  Navy  and  its  jobs  from 
those  in  the  civilian  job  market.  This  uniqueness  feature  changes  over  time  and 
is  illustrated  by  the  various  slogans  which  have  been  the  backbone  of  national 
advertising  campaigns. 

D.  NAVY  SLOGANS 

The  primary  slogan  in  use  from  1976  to  1986  was  "It's  Not  Just  a  Job,  It's 
an  Adventure."  The  USP  of  this  slogan  was  the  extra  dimension  of  adventure, 
placing  an  emphasis  on  convincing  young  men  (at  the  time,  all  advertising  was 
focused  on  recruiting  1 8-24  year  old  men)  that  the  Navy  provided  jobs  and 
opportunities  that  were  superior  to  their  civilian  counterparts.  Although  no 
longer  the  official  slogan,  there  are  still  publications  and  brochures  in  recruiting 
offices  imprinted  with  the  "It's  Not  Just  a  Job..."  slogan. 

The  slogan  which  followed,  "You  Are  Tomorrow,"  served  as  the  official 
slogan  for  only  two  years,  from  1987  through  1988.  While  short-lived  in  terms  of 
exposure  time,  polls  have  shown  it  to  be  more  readily  identified  to  the  Navy  than 
its  predecessor,  which  is  often  accredited  to  the  Army.  (Lerro,  1991 ) 
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The  current  official  slogan  is  "You  and  the  Navy,  Full  Speed  Ahead,"  with 
extra  emphasis  being  place  on  the  "You."  This  slogan  has  been  in  place  since 
1 989  and  intends  to  convey  the  message  that  the  Navy  cannot  forge  on  into  the 
future  without  the  men  and  women  who  make  up  its  ranks. 

E.  MEDIA  UTILIZATION 

The  general  media  utilization  policy  of  the  Navy’s  national  advertising 
program  is  that  peak  advertising  should  occur  in  the  periods  of  April  to  May  and 
September  to  November.  This  policy  is  based  on  the  premise  that  critical 
decision  periods  for  the  primary  target  group  occur  prior  to  graduation  in  the  late 
spring,  and  then  again  in  the  fall.  The  distribution  of  resources  is  also  driven  by 
the  characteristics  of  the  different  media. 

The  various  types  of  media  in  the  national  advertising  program  are 
television,  radio,  magazines,  direct  mail,  newspapers,  supplemental,  and 
outdoor.  The  distribution  of  resources  to  each  is  determined  by  policies  and 
practices  pertaining  to  the  type  of  media  utilized. 

Television  allows  an  advertising  theme  to  benefit  from  the  sight,  sound, 
and  motion  employed  by  this  medium.  Though  time  for  syndicated,  network,  and 
cable  television  programs  must  be  purchased  well  in  advance  of  the  appearance 
date,  television  maintains  relatively  high  audience  selectivity.  Thus,  it  is 
probably  the  best  suited  advertising  medium  for  increasing  awareness  in  a  broad 
reach  capacity. 
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Paid  radio  has  a  high  frequency  of  impact  and  a  fast  reach  capability.  It 
provides  a  flexible  means  of  advertising  without  requiring  the  long  lead  time  of 
television.  In  addition,  radio  advertising  provides  the  ability  to  be  quite  selective 
in  the  target  audience. 

Magazines  provide  a  continuous,  year-round  program  that  transmits  an 
advertisement  in  detail.  Magazine  advertising  has  a  relatively  long  lag  effect 
since  it  is  hard  copy  and  retained  for  extended  periods  of  time.  Inquiries  based 
on  magazine  ads  continue  to  be  received  weeks,  months,  and  occasionally, 
years  after  the  insertion  of  an  ad.  The  increase  in  subject-focused  magazines 
(i.e.,  professional  journals,  "teen"  magazines,  etc.)  provides  fairly  high  audience 
selectivity.  Magazines  facilitate  leads  generation  with  the  insertion  of  pop-up 
cards  or  on-page  coupons  (Arima,  1978). 

Direct  mail  is  an  especially  versatile  medium  because  it  allows  the  most 
specificity  and  segmentation  of  the  target  audience  (i.e.,  this  year's  graduating 
seniors  in  predominantly  black  schools)  (Arima,  1978).  Its  mail-back  postcard 
feature  makes  it  a  successful  leads  generator  and  an  action-oriented  device 
rather  than  simply  a  tool  for  influencing  awareness  or  attitudes. 

Newspapers  and  supplementals  are  not  widely  used  on  the  national  level. 
The  primary  use  would  be  to  rectify  shortages  in  critical  occupational  areas. 
There  has  been  no  money  spent  on  a  national  newspaper  advertisement  since 
second  quarter  FY  1992.  Since  then,  only  the  third  quarter  of  FY  1993  included 
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supplemental  advertisement  funding.  Newspapers  and  supplementals  are 
primarily  for  local  recruitment  advertising. 

Over  a  decade  ago,  outdoor  advertising,  such  as  billboard  murals,  was 
used  rather  extensively  for  limited  purposes.  While  the  potential  for  broad 
coverage  is  great,  the  problem  found  with  outdoor  advertising  was  that  the 
viewer  was  usually  in  rapid  motion.  This  does  not  facilitate  a  good  transmission 
of  information.  Thus,  outdoor  advertising  on  a  national  level  has  not  received 
any  funding  in  the  past  decade. 

F.  NAVY  ADVERTISING  OBJECTIVES 

The  Annual  Advertising  Plan  (ADPLAW)  of  the  Marketing  Communications 
Department  sets  the  marketing  objectives  for  each  recruiting  program.  Since 
advertising  is  a  part  of  the  overall  marketing  plan,  the  advertising  program 
should  contribute  toward  the  achievement  of  the  marketing  objectives.  However, 
the  difference  between  advertising  and  marketing  is  sometimes  hard  to 
distinguish. 

Advertising  objectives  listed  in  the  current  ADPLAN  for  the  general 
enlisted  program  are  quite  comprehensive.  Preceding  the  list  of  objectives  is  an 
assertion  stating  that  objectives  are  to  be  achieved  by  building  a  greater 
awareness  of,  and  positive  attitude  toward,  the  Navy.  The  list  of  specific 
objectives  is  as  follows: 

1 .  Greater  walk-in  traffic  at  the  recruiting  station. 

2.  More  favorable  response  to  a  recruiter's  telephone  call. 
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3.  Increased  calls  to  the  toll-free  number. 

4.  More  leads  generated  to  NOIC. 

5.  Greater  discussion  among  influentials. 

6.  More  favorable  reception  to  recruiter's  request  to  visit  groups. 

7.  Increased  dialogue  between  prospect  and  recruiter. 

8.  Better  understanding  of  Navy  benefits. 

9.  Greater  receptivity  to  direct  mail  solicitations. 

1 0.  Greater  respect  for  the  Navy  recruiter  within  the  community. 

1 1 .  Increased  self-esteem  among  the  field  recruiters 

12.  The  understanding  by  the  prospect  that  jobs  and  careers  in  the  Navy 
are  more  rewarding  than  those  in  civilian  jobs.® 

It  should  be  noted  that  the  primary  objective  of  the  Navy  national  advertising 
program  is  to  enhance  awareness  among  the  population.  This  differs  from  the 
purpose  of  local  advertising,  which  is  to  attain  sufficient  quantities  of  qualified 
recruits.  As  such,  the  problem  lies  in  the  identification  of  a  measurable  means 
for  evaluating  the  success  of  the  national  advertising  campaign. 

G.  YOUTH  ATTITUDE  TRACKING  STUDY  (YATS) 

YATS  is  an  annual  survey  of  approximately  10,000  men  and  women,  aged 
16  to  24.  Respondents  are  identified  through  a  random  selection  of  telephone 
numbers.  Computer  assisted  telephone  interviews  (CATIs)  collect  information 
on  backgrounds,  current  status  and  future  plans,  self-reported  likelihood  of 


®  The  listed  objectives  are  part  of  the  FY  1990  Annual  Advertising  Plan  (ADPLAN)  and  are  still  in 
effect. 
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enlisting  in  the  military,  and  perceptions  concerning  a  wide  variety  of  relevant 
issues.  CATI  presents  questions  on  a  computer  screen  to  the  interviewer  to 
read  over  the  telephone,  and  interviewers  type  the  responses  into  a  database  as 
the  interview  is  conducted.  This  technology  eliminates  inappropriate  questions 
based  on  the  respondent's  earlier  answers  and  identifies  inconsistent  responses 
during  the  interview.  (Lerro,  1991) 

YATS  is  designed  to  help  determine  how  effective  military  advertising  is  in 
increasing  awareness  among  its  target  audience.  Data  on  service  advertising 
awareness  levels,  slogan  recognition,  reactions  to  receipt  of  service  literature, 
and  actions  taken  to  obtain  information  about  the  military  are  analyzed  in  terms 
of  demographics,  advertising  exposure,  and  military  perceptions. 

H.  NAVY  NEW  RECRUIT  SURVEY  (NRS) 

The  NRS  is  administered  to  new  recruits  at  Recruit  Training  Centers  three 
times  a  year.  The  NRS  was  administered  in  1990-1991  but  was  subsequently 
suspended  until  1994.  It  has  since  been  coordinated  by  CNRC  Code  22  and 
administered  in  July  1994,  November  1994,  April  1995,  and  August  1995.  The 
NRS  is  designed  to  assess  the  effectiveness  of  recruiting  incentives,  advertising, 
and  applicant  processing.  Several  content  areas  are  addressed  in  the  survey: 
recruiter  contact,  recruiting  advertising,  enlistment  motivation,  job  choices  or 
preferences.  Navy  College  Fund,  employment  history,  and  magazine 
preferences. 
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The  survey  includes  questions  pertaining  to  "core"  items,  which  remain 
constant  over  time,  as  well  as  "current"  items  that  would  be  responsive  to  special 
programs  or  immediate  concerns  (Robertson,  1993).  The  "core"  item  questions 
serve  to  track  trends  in  areas  such  as  demographics,  backgrounds,  and 
enlistment  motivation.  The  "current"  items  might  change  with  each  survey 
according  to  world  events,  such  as  the  influence  of  the  Gulf  War,  or  current 
popular  media  programs,  such  as  movies  or  popular  television  shows.  In  this 
capacity,  the  magazine  preference  section  aids  in  determining  the  magazine 
advertising  plan  by  revealing  the  most  popular  titles  among  the  target  audience. 
Results  of  the  NRS  assist  in  evaluating  the  effectiveness  of  national  advertising 
in  reaching  its  target  audience,  increasing  awareness,  and  promoting  a  positive 
attitude  toward  the  Navy. 
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III.  ESTIMATING  ADVERTISING  COST-EFFECTIVENESS 


A.  ADVERTISING  POLICY  ISSUES 

The  cost  of  advertising  is  difficult  to  justify  in  both  the  public  and  private 
sectors.  The  problem  of  measuring  advertising  effectiveness  becomes  key  due 
to  an  incapacity  to  predict  the  return  on  investment  for  advertising  expenditures. 
It  is  difficult  to  refute  the  usefulness  of  advertising  in  the  marketing  effort. 
However,  justifying  budget  requirements  and  demonstrating  the  relationship  of 
expenditures  to  outcomes  become  the  primary  issues. 

1.  Advertising  Goals 

Establishing  an  effective  allocation  of  resources  means  that  policymakers 
must  address  a  series  of  questions  concerning  the  goals  and  methods  of 
advertising  programs.  A  target  audience  must  be  identified,  a  budget 
established,  dollars  must  be  allocated  among  alternative  media,  and  an  optimal 
pattern  of  expenditures  must  be  implemented  over  time.  This  optimal  allocation 
of  expenditures  depends  on  the  objectives  of  advertising  which  may  vary  in 
different  circumstances. 

In  the  case  of  recruiting  advertising,  the  current  accession  status  may 
denote  a  particular  strategy.  During  an  economic  decline,  enlistment 
propensities  may  be  adequate  without  any  advertising.  In  such  an  environment, 
an  appropriate  campaign  may  emphasize  building  a  foundation  for  future 
enlistments  by  targeting  a  younger  population.  In  contrast,  a  shortage  of 
accessions  in  any  given  occupational  specialty  may  impel  an  advertising 
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campaign  consisting  of  a  series  of  advertisements  in  local  newspapers  or  high 
school  publications  emphasizing  enlistment  bonuses  or  educational  benefits. 
(Dertouzos  and  Polich,  1989) 

DoD  management  may  have  differing  goals;  it  may  be  concerned  with 
public  image  over  the  long  term,  assuming  that  by  enhancing  the  public's  view  of 
the  military,  the  recruiting  and  retention  climate  for  the  military  and  civilian 
employees  will  improve.  A  clearly  stated  advertising  goal  helps  justify 
advertisement  budget  requirements. 

2.  Establishing  a  Budget 

Webb  and  Sohn  (1994,  p16)  provide  the  following  stages  for  preparing  an 
advertising  budget.  They  entail  costing  out  the  advertising  elements  required  to 
increase  sales.  The  stages  of  preparing  the  budget  are: 

(1)  Define  and  evaluate  a  "target  audience"  in  terms  of  the  types 
of  media  and  frequency  of  advertising  exposure  needed  to  elicit 
responses  leading  to  sales. 

(2)  Decide  how  many  and  what  kinds  of  advertisements  to  run  for 
each  media  program  size  and  type. 

(3)  Estimate  the  cost  of  creating  and  maintaining  an  advertisement 
pool  that  supports  the  number  and  type  of  advertisements 
selected.  Note  that  complexity  of  the  product  offer  and  the 
receptivity  of  the  audience  influences  effectiveness  of  the  pool. 

(4)  Cost  out  a  national  media  budget  that  will  provide  the  desired 
level  of  continuing  exposure.  Media  budget  size  is  critical:  an 
individual  advertisement  will  not  cause  awareness  if  not  seen  or 
heard  enough  times  to  affect  awareness.  Thus,  the  budget  size 
limits  the  size  of  the  advertisement  pool.  There  also  exists 
evidence  that  "wear  out"  occurs  from  overexposing  the  target 
audience  to  the  same  advertisements.  "Wear  out"  provides  a 
rationale  for  expanding  the  number  and  type  of  advertisements  run. 
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(5)  Determine  costs  of  related  essential  operational  elements  such 
as  a  system  for  responding  to  inquiries,  processing  leads,  and 
recording  data  needed  to  show  advertising  cost-effectiveness. 

Although  this  method  of  establishing  a  budget  is  outwardly  logical, "...  the 

verification  of  specific  underlying  assumptions,  including  effectiveness  and 

costs,  is  difficult"  (p1 6). 

3.  Allocation  of  Dollars  Among  Alternative  Media 

Dertouzos  and  Polich  (1989)  suggest  that "...  the  size  of  the  advertising 
budget  may  be  viewed  as  being  fixed  in  the  short  run.  However,  for  the 
organization  as  a  whole,  advertising  represents  one  type  of  resource  that  must 
compete  with  alternative  expenditures.  (p4)" 

When  given  an  overall  budget,  managers  must  evaluate  the  many  competing 
media  available  for  delivering  messages  to  the  target  audience.  CNRC 
allocates  its  advertising  budget  between  national  and  local  advertising,  and 
across  national  media  type.  For  national  advertising,  CNRC  and  their 
advertising  agency  (BBDO)  determine  howto  divide  the  disbursements  across 
NRAs  and  NRDs  in  order  to  effectively  reach  the  target  audience. 

Advertising  purchases  can  differ  by  the  size  and  demographics  of  the 
intended  audience,  the  cost  of  delivery,  and  potential  penetration  of  the  target 
market.  In  addition,  alternative  media  may  have  comparative  advantages  in 
conveying  qualitative  messages.  For  example,  prime-time  network  television  is 
likely  to  be  more  efficient  at  exposing  varied  national  audiences  to  messages 
that  convey  broad  images,  while  display  advertisement  in  a  high  school 
newspaper  can  provide  detailed  information  to  a  distinct  demographic  segment 
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of  the  population  (Dertouzos  and  Polich,  1989).  Depending  on  the  objectives  of 
advertising,  each  medium  has  potential  to  play  a  different  role. 

B.  MEASURING  AWARENESS 

Navy  national  advertising  enhances  the  image  of  the  Navy  in  the  public 
mind,  creating  awareness  about  the  Navy,  its  benefits,  and  its  opportunities.  It 
assumes  that  such  awareness  will  improve  the  recruiting  climate  and  enhance 
the  retention  of  enlistees.  Unfortunately,  this  "awareness"  is  not  readily 
measurable.  CNRC  can  refer  to  indications  of  high  awareness  of  campaign 
slogans  among  youth  as  recorded  in  tracking  surveys,  such  as  YATS,  and  assert 
that  advertising  creates  awareness.  However,  it  cannot  prove  that  changes  in 
advertisement  exposure  achieved  corresponding  changes  in  the  levels  of 
awareness. 

Awareness  is  usually  equated  to  exposure,  or  a  respondent's  knowledge 
about  the  Navy,  its  benefits,  and  its  opportunities.  It  is  a  function  of 
expenditures  on  media,  direct  mail,  and  local  advertising,  along  with  other 
factors,  such  as  family  members  who  have  joined  the  service.  Awareness  of  the 
benefits  of  military  service  may  lead  a  young  person  to  respond  positively  to  a 
call  from  a  recruiter  or  some  other  independent  stimulus. 

Awareness  is  currently  measured  by  advertising  impressions.  Advertising 
agencies  provide  estimates  of  the  number  of  impressions  gathered  each  month 
by  the  agency's  advertising  on  television,  radio,  and  magazines  and  through 
direct  mail  campaigns.  An  impression  is  defined  as  the  exposure  of  one 
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individual  to  one  advertisement.  The  total  number  of  impressions  is  the  number 
of  times  an  advertisement  is  run  (frequency)  times  the  number  of  people  who 
see  the  advertisement  (audience  size).  Because  they  are  said  to  be  the  best 
agent  for  capturing  the  effect  of  an  advertisement  campaign  on  potential 
enlistees,  their  families  and  friends,  impressions  are  used  as  a  measurement  of 
awareness  or  advertising  effectiveness. 

The  problem  associated  with  the  use  of  impressions  as  a  measurement  of 
awareness  lies  in  the  area  of  the  unknown.  Impressions  measure  "potential" 
viewers  based  on  projected  audience  sizes.  There  is  no  concrete  manner  to 
measure  the  actual  number  of  viewers  of  any  advertisement  medium  at  any 
given  time.  Direct  mail  comes  the  closest  with  knowledge  of  the  number  of 
mailings  sent  out  and  delivered.  However,  it  remains  impossible  to  determine 
the  exact  number  of  individuals  who  actually  opened  and  read  the  mailing. 

This  study  makes  an  argument  for  using  the  number  of  leads  generated 
as  a  measure  of  awareness.  The  number  of  leads  generated  is  measurable 
down  to  the  recruiting  district  level.  The  number  of  phone  calls  into  the  toll-free 
number  is  also  measurable  at  the  recruiting  district  level.  The  number  of  phone 
calls  may  be  indicative  of  the  number  of  positive  impressions  made  to  the  target 
audience  through  national  advertising  campaigns.  Leads  generation  costs  and 
awareness  costs  are  usually  differentiated  as  being  marketing  and  advertising 
costs,  respectively.  While  the  bridge  between  "sales"  and  awareness  is  seldom, 
if  ever,  crossed  by  national  advertising  agencies,  it  is  the  intent  of  this  study  to 
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establish  a  link  between  the  two  in  order  to  determine  the  true  effectiveness  of 
the  Navy's  advertising  expenditures. 

C.  DATA 

1.  Budget 

U.S.  Navy  Budget  and  Expenditure  (B&E)  reports  for  contracted 
advertising  campaigns  were  compiled  for  a  three-year  period,  from  FY  1993 
through  FY  1995.  The  B&E  reflects  CNRC's  budgeted  production  and  execution 
dollars  as  well  as  overhead,  general  and  administrative,  postage,  and 
miscellaneous  amounts  for  contracted  television,  magazine,  radio,  and  direct 
mail  national  advertising. 

Full  advertising  contracts  are  forged  in  an  18-month  cycle.  This 
incorporates  negotiation  of  a  fixed  price  contract,  setting  of  appearance  dates, 
requests  for  bids,  contract  award,  production,  and  execution.  The  B&E  indicates 
the  anticipated  appearance  date  for  each  specific  advertisement.  There  is  a  six- 
month  lag  effect  built  into  this  date.  The  dollars  corresponding  to  the 
appearance  date  were  authorized  in  the  budget  dated  six  months  prior.  Thus, 
the  B&E  is  continuously  updated  to  reflect  changes  in  production  schedules, 
appearance  dates,  canceled  advertisements,  and  emergency  requirements. 

The  B&E  categorizes  expenditures  by  advertising  medium  (television, 
magazine,  radio,  direct  mail),  then  by  majority  and  minority  (Hispanic  and  Black) 
groups,  and  finally  into  target  occupations  or  audiences  (Nuclear  Officer, 
NROTC,  General  Enlisted,  etc.).  Other  expenditures  relating  to  overhead. 
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general  and  administrative,  and  postage  are  also  indicated.  Only  data  for  the 
general  enlisted  program  were  used  in  this  study. 

2.  Advertising  Expenditures 

Actual  national  advertising  expenditures  for  the  general  enlisted  program 
were  collected  for  FY  1993  through  the  second  quarter  of  FY  1995  from  the  NRD 
Allocation  Matrix  and  the  Recruit  Information  Delivery  System  (RIDS).  Figures  in 
the  NRD  Allocation  Matrix  are  maintained  by  the  advertising  agency,  BBDO,  and 
categorized  by  month  by  media  type,  by  NRD.  RIDS  is  maintained  by  Code  22, 
the  Research  and  Analysis  Division,  of  CNRC.  The  expenditures  listed  within  it 
mirror  those  in  the  NRD  Allocation  Matrix. 

3.  Impressions 

Impressions  data  for  Navy-unique,  general  enlisted  advertisements  were 
also  collected  for  FY  1993  through  the  second  quarter  of  FY  1995  from  the  NRD 
Allocation  Matrix  and  RIDS.  These  figures  were  categorized  by  month  by 
medium  by  NRD  for  a  target  audience  of  18-24  year  old  males. 

4.  Leads  Generated 

National  general  enlisted  leads  generated  figures  were  collected  for  FY 
1993  through  FY  1995  from  the  Navy  Advertising  Leads  Tracking  System 
(NALTS).  These  data  were  classified  by  month  by  NRD  and  were  an  aggregate 
of  television,  radio  and  magazine  leads. 
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5.  New  Recruit  Surveys  (NRS)  and  Youth  Attitude  Tracking  Survey 
(YATS) 

Copies  of  the  NRSs  and  corresponding  answers  were  collected  from  the 
July  1994,  November  1994,  April  1995,  and  August  1995  surveys.  Answers  to 
the  advertising  awareness  questions  were  compiled  and  analyzed  to  determine 
the  level  of  influence  advertising  possessed  on  a  new  recruit's  decision  to  enlist 
and  to  examine  the  changes  in  advertising  awareness  among  the  target 
audience.  Data  on  propensities  to  enlist  as  determined  by  the  YATS,  were 
collected  for  males  and  females  for  FY 1993  through  FY  1995. 

6.  Miscellaneous  Data 

Unemployment  rates  by  month  and  by  NRD  were  collected  via  the  RIDS 
for  FY  1 993  through  the  second  quarter  of  FY  1 995. 

D.  DATA  LIMITATIONS 

1.  Budget  Data 

CNRC's  budget  is  separated  into  two  portions:  contracted  and  in-house 
advertising  expenditures.  The  B&E  reflected  only  contracted  advertising 
expenditures  which  included  all  national  advertising  disbursements.  Because  it 
did  not  include  in-house  advertising  costs  (RAD  items  and  field  dollars), 
corroboration  between  execution  expenditures  listed  in  the  B&E,  NRD  Allocation 
Matrix  and  RIDS  is  impossible.  Therefore,  the  B&E  could  only  be  used  to 
determine  production  expenditure  figures. 
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The  B&E  does  not  facilitate  the  "snapshot"  computation  of  accurate 
spending  levels  at  any  given  point  in  time.  Because  of  the  length  of  time 
required  to  complete  a  contract,  figures  in  the  B&E  are  continually  changing  to 
account  for  contract  changes,  canceled  campaigns,  schedule  changes,  etc.  until 
well  after  the  appearance  date.  Thus,  even  year-end  B&E  reports  dated  on 
September  30th  are  not  necessarily  accurate  unless  all  budget  changes  have 
been  included.  Additionally,  because  the  B&E  is  the  only  record  of  production 
expenditures,  the  accuracy  of  these  figures  is  questionable  without  verification 
that  all  contract  changes  have  been  explicated. 

The  anticipated  appearance  dates  in  the  B&E  do  not  correspond  to  the 
budget  dates.  Advertising  expenditures  are  listed  according  to  the  fiscal  year  of 
their  appearance  date,  however,  they  were  funded  by  the  budget  in  effect  six 
months  prior  to  that  date.  This  approximate  lag  effect  does  not  facilitate  an 
accurate  accountability  of  the  budget. 

2.  Advertising  Expenditure  Data 

The  NRD  Allocation  Matrix  and  RIDS  only  reflect  execution  expenditures 
and  do  not  serve  as  sources  of  production  cost  information.  In  addition,  only 
expenditure  information  through  the  second  quarter  of  FY  1995  was  available 
from  either  database.  While  all  advertising  execution  figures  were  obtained  by 
media  type,  month  and  NRD,  production  amounts  from  the  B&E  were 
aggregated  into  year-end  totals.  Thus,  in  order  to  compare  production  overhead 


33 


to  execution  dollars,  the  execution  figures  had  to  be  aggregated  to  match  those 
from  the  B&E,  resulting  in  limited  findings. 

3.  Leads  Generated  Data 

National  leads  data  maintained  by  the  NALTS  do  not  distinguish  between 
media  type  which  generated  the  lead.  The  only  leads  data  were  an  aggregate  of 
television,  radio  and  magazine  leads.  Additionally,  national  leads  resulting  from 
direct  mail  are  not  maintained  in  any  database.  The  only  national  direct  mail 
data  are  annotated  in  the  NALTS  Summary  Report  by  Agency  which  allocates 
responses  into  the  toll-free  number  to  specific  agencies.  MARCOA,  the  agency 
which  handles  national  direct  mail,  is  attributed  with  leads  through  this  means; 
however,  there  is  no  evidence  that  the  responses  listed  in  this  report  account  for 
all  possible  leads  generated.  Many  leads  are  not  generated  by  a  telephone  call 
to  the  toll  free  number,  and  thus  are  not  included  in  the  NALTS  Summary 
Report. 

The  aggregated  leads  data  were  maintained  by  month  by  NRD;  however, 
prior  to  FY  1995,  current  month  leads  were  not  directly  tracked.  Instead,  "total 
leads  to  date"  were  compiled  which  included  carry  over  leads  from  prior  fiscal 
years  as  well  as  total  fiscal  year  to  date  leads.  This  required  the  conversion  of 
the  "total  leads  to  date"  into  current  month  leads.  Beginning  in  FY  1995,  current 
month  leads  were  maintained  and  accounting  of  carry  over  leads  was 
discontinued. 
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IV.  ANALYSIS 


A.  STUDY  METHODOLOGY 

This  study  uses  data  collected  from  CNRC  which  was  entered  into  a 
database  for  analysis.  Year-end  (fiscal  year)  B&E  reports  provide  advertising 
production  expenditures.  To  properly  allocate  money  to  the  correct  fiscal  year, 
this  study  uses  a  six  month  lag  effect  from  the  expected  view  date  to  determine 
production  expenditures.  For  example,  payment  for  a  magazine  insert  with  a 
view  date  of  January  1996  was  made  from  the  FY  1995  budget  (six  months  prior 
to  the  view  date).  This  thesis  determines  television,  radio,  magazine  and  direct 
mail  production  figures  similarly,  and  compares  them  to  the  corresponding 
execution  expenditures  derived  from  the  NRD  Allocation  Matrix  and  RIDS.  As 
previously  mentioned,  the  accuracy  of  the  B&E  production  figures  are 
questionable  due  to  the  length  of  time  required  to  fulfill  an  advertising  contract. 
The  year-end  reports  are  irresolute  and  it  cannot  be  verified  that  all  contracts 
listed  in  the  report  have  been  fulfilled  and/or  adjusted  as  required. 

In  contrast  to  the  B&E,  the  NRD  Allocation  Matrix  facilitates  the 
determination  of  accurate  execution  expenditures  for  the  varying  media  types. 
BBDO  accounts  for  all  advertising  changes  and  reflects  them  in  the  numbers 
they  list  in  the  matrix.  Additionally,  these  figures  are  verifiable  with  those  listed 
in  the  RIDS. 

The  advertising  execution  expenditures  and  impressions  data  were 
compiled,  by  media  type,  from  the  RIDS  and  the  NRD  Allocation  Matrix.  This 
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information  includes  television,  radio,  magazine,  direct  mail,  newspaper,  outdoor 
and  supplemental  figures.  All  were  entered  into  the  database  by  NRD  and 
month  for  FY  1993  through  the  second  quarter  of  FY  1995.  Files  aggregating 
the  television,  radio  and  magazine  expenditures  and  impressions  were  created 
to  facilitate  correlation  with  the  leads  data.  Unemployment  rate  information  for 
each  NRD  by  month  was  also  compiled  into  the  same  database  for  FY  1993 
through  the  second  quarter  of  FY  1 995. 

This  study  then  compiles  monthly  national  leads  data  for  FY  1993  through 
FY  1995  from  a  hard  copy  of  the  NALTS  Executive  Summary  Reports  for  each 
NRD.  These  leads  are  an  aggregate  of  those  generated  by  national  television, 
radio  and  magazine  advertising.  No  national  direct  mail  leads  information  was 
available.  As  previously  mentioned,  CNRC  did  not  maintain  leads  information 
prior  to  FY  1995  on  a  current  month  basis  (i.e.,  number  of  new  leads  generated 
in  a  given  month).  Instead,  CNRC  reported  "total  leads  to  date"  which  contained 
carry  over  leads  from  prior  fiscal  years  along  with  current  year  leads.  This 
required  the  conversion  of  the  "total  leads  to  date"  into  current  month  leads  for 
FY  1993  and  FY  1994. 

A  second  conversion  was  required  of  the  leads  data.  Each  year,  CNRC 
reviews  the  recruiting  districts  and  geographic  areas.  This  review  frequently 
results  in  the  shifting  of  boundaries,  requiring  some  NRDs  to  be  realigned, 
renamed  or  combined  within  other  NRDs.  Effective  in  FY  1995,  CNRC  dissolved 
NRA  seven,  and  a  major  realignment  of  the  NRDs  took  place.  Because  of  this 
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change,  this  study  converts  the  leads  data  from  FY  1993  and  FY  1994  NRDs  to 
FY  1995  NRDs  for  consistency  in  comparison  and  correlation.  All  other 
information  received  from  the  RIDS  accounted  for  the  change.  Derivation  of  the 
following  correlations  occured  upon  completion  of  the  database; 

1 .  Impressions  versus  dollars  by  media  type  for  each  NRD  and  national 

totals. 

2.  Dollars  versus  aggregated  television,  radio  and  magazine  leads  for 
each  NRD  and  national  totals. 

3.  Impressions  versus  aggregated  television,  radio  and  magazine  leads 
for  each  NRD  and  national  totals. 

4.  Unemployment  rate  versus  leads  for  each  NRD  and  national  totals; 
This  study  graphically  presents  the  correlations  to  illustrate  seasonal  trends  in 
expenditures,  impressions  and  leads.  The  study  focuses  on  four  NRDs:  102 
(New  England),  313  (Atlants),  547  (ST.  Louis)  and  836  (Los  Angeles).  These 
four,  as  case  studies,  represent  the  data  in  general. 

Finally,  New  Recruit  Survey  information  was  compiled  from  the  July  1994, 
November  1994,  April  1995  and  August  1995  surveys.  A  tally  of  respondent's 
answers  to  the  advertising  awareness  questions  led  to  an  investigation  of 
patterns  which  might  reveal  shifting  attitudes  towards  the  Navy. 
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Table  3 

General  Enlisted  Contracted  Advertising  Expenditures 


Production  Overhead®  Execution^ 


Television  Television 


FY1993 

142503 

FY  1993 

301750 

FY1994 

1237262 

FY1994 

4402793 

FY1995 

1491443 

FY  1995(2nd  qtr) 

8764659 

Radio 

FY1993 

178863 

Radio 

FY  1993 

1452936 

FY1994 

340843 

FY  1994 

2891864 

FY1995 

200543 

FY  1995(2nd  qtr) 

1488725 

Magazine 

FY1993 

0 

Magazine 

FY  1993 

1044899 

FY  1994 

8668 

FY  1994 

1575233 

FY  1995 

0 

FY  1995(2nd  qtr) 

1234095 

Direct  Mail 

FY1993 

0 

Direct  Mail 

FY1993 

761064 

FY1994 

0. 

FY  1994 

1253274 

FY1995 

0 

FY  1995(2nd  qtr) 

994758 

®  Production  overhead  figures  based  on  annual  CNRC  Budget  and  Expenditures  (B&E)  Report. 
FY  1995  figures  as  of  August  1995.  ^ 

^  Execution  figures  based  on  NRD  Allocation  Matrix.  FY  1995  figures  only  through  March  1995 
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B.  RESULTS 


Table  3  shows  production  and  execution  expenditures  for  general  enlisted 
national  advertising  (television,  radio  and  magazine).  Sharp  increases  reflect 
plus-ups  in  the  Navy's  advertising  budget  (in  both  production  and  execution 
figures)  from  FY  1993  to  FY  1994.  Additionally,  even  though  only  the  first  two 
quarters  are  reported  for  FY  1995,  there  continues  an  upward  trend  in  the  the 
national  advertising  budget.  Television  execution  expenditures  for  the  first  half 
of  FY  1 995  are  more  than  twice  the  FY  1 994  year-end  figures.  The  release  of 
several  new  advertisement  campaigns  in  FY  1995  partly  account  for  this 
increase. 

Although  the  overall  dollar  increases  are  relatively  obvious,  possible 
inaccuracies  in  the  B&E  prevent  a  more  in-depth  analysis.  To  provide  more 
precise  analysis  would  require  verification  of  expenditures  and  more  accurate 
definition  of  advertising  programs.  Additionally,  similar  data  limitations  exist  in 
comparing  the  budget  to  the  number  of  impressions  purchased  and  leads 
generated.  Because  the  leads  data  were  an  aggregate  of  television,  radio  and 
magazine  leads,  dollar  and  impression  figures  had  to  be  aggregated  for 
comparison. 

Figure  4  represents  a  comparison  of  total  national  advertising 
expenditures  (television,  radio  and  magazine)  to  corresponding  total  impressions 
and  total  leads.  Once  again,  an  impression  is  defined  as  the  exposure  of  one 
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Figure  4.  General  Enlisted  National  Advertising  Totals 
Aggregated  Television,  Radio  and  Magazine  Data 
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TOTAL  leads  —TOTAL  dollars 


individual  to  advertisement.  Since  impressions  are  largely  determined  by  the  be 
number  of  dollars  spent,  the  correlation  between  dollars  and  impressions  should 
very  close  to  one  (i.e.,  perfect  correlation).  However,  examination  of  the 
relationship  between  monthly  aggregated  dollars  and  impressions  from  FY  1993 
second  quarter  of  FY  1995,  revealed  a  correlation  of  0.78077.  Based  on  the 
definition  of  an  impression,  the  aggregate  data  does  not  accurately  depict  the 
relationship  of  impressions  to  dollars. 

Results  of  the  four  case  studies  further  illustrate  this  point.  When 
examined  at  the  district  level,  by  month,  and  by  media  type,  comparison  of 
impressions  versus  dollars  resulted  in  the  following  correlations; 


Television 

Radio 

Magazine 

NRD  102 

.94822 

.98713 

.91800 

NRD  313 

.96384 

.96363 

.95350 

NRD  547 

.95962 

.91172 

.94817 

NRD  836 

.95193 

.99124 

.91926 

These  figures,  specified  by  media  type,  are  much  closer  to  what  one  would 
expect  the  correlations  to  be.  Figures  5  through  8  graphically  illustrate  how 
impressions  and  dollars,  by  media  type,  moved  together  over  the  three  years 
examined  for  each  of  the  NRDs. 

Correlations  between  leads  and  dollars  showed  no  real  relationship, 
either  with  aggregated  national  totals  or  when  specified  by  NRD.  For  national 
totals,  the  correlation  was  estimated  at  0.1 1 1 1 3,  displaying  no  strong  association 
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Figure  5.  NRD 102  Impressions  vs.  Dollars  by  Media  Type 
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Figure  7.  NRD  547  Impressions  vs.  Dollars  by  Media  Type 
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in  either  the  positive  or  negative  direction.  Figure  4  depicts  this  absence  of  any 
real  association.  Leads  appear  relatively  independent  of  the  amount  of  money 
spent  in  a  given  month  on  national  advertising. 

When  the  data  were  specified  by  NRD,  the  four  case  studies  revealed  the 
same  lack  of  conclusive  association  between  dollars  and  leads: 

Correlation  with  leads 
NRD  102  .06720 

NRD  313  .19141 

NRD  547  .02606 

NRD  836  .19445 

Because  no  relationship  could  be  substantiated,  the  number  of  leads  generated 
were  shifted  up  to  six  months  determine  if  advertisement  had  a  possible  "lag 
effect"  on  the  leads.  This  resulted  in  the  following  correlations  between  leads 
and  dollars: 

Lead  Lag  Time 


1  mo 

2  mos 

3  mos 

4  mos 

5  mos 

6  mos 

National 

.04051 

.07979 

.31810 

.34563 

.13664 

-.07059 

Totals 

NRD  102 

-.2943 

.0088 

.42307 

.44991 

-.02788 

-.04312 

NRD  313 

-.23312 

-.06774 

.42795 

.47522 

-.03883 

-.15410 

NRD  547 

-.32822 

-.14684 

.41496 

.54565 

.17818 

-.14092 

NRD  836 

-.0453 

.09455 

.58395 

.52099 

.39931 

.02183 
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While  there  is  still  no  evidence  of  strong  correlation,  it  is  clearly  evident 
that  the  correlations  seemed  to  be  the  strongest  with  a  three-  or  four-month  lag 
effect.  This  was  true  for  the  national  totals  as  well  as  when  broken  into  NRDs. 
The  correlations  showed  characteristics  of  strengthening  with  each  additional 
lagged  month  until  the  three-  or  four-month  point.  After  this,  they  all  dropped 
significantly.  Figures  9  through  12  illustrate  the  difference  between  leads  versus 
dollars  before  any  lag  effect  and  with  a  four-month  lag  effect  in  place  for  the  four 
case  study  NRDs.  It  can  be  hypothesized  that  advertising  has  a  four  month  lag 
effect  on  leads  generated;  however,  in  order  to  further  analyze  this  hypothesis,  a 
regression  analysis  with  data  broken  down  by  month,  by  NRD,  and  by  media 
type  is  required  to  substantiate  any  potential  lag  effect. 

It  is  also  important  to  note  that  the  aggregated  (television,  radio  and 
magazine)  national  data  misrepresents  many  of  the  NRDs.  The  correlation 
between  leads  and  dollars  using  the  aggregated  information  deviates  both 
positively  and  negatively  from  the  NRD  specific  information,  further  evidence 
that  using  aggregated  data  for  a  cost-effectiveness  analysis  produces 
suboptimal  findings. 

Analysis  of  the  leads  data  over  the  three  year  period  demonstrates  a 
pattern  in  inquiries  about  the  Navy  which  may  be  independent  of  any  advertising 
program.  There  are  distinct  “peaks”  in  the  volume  of  leads  generated  several 
times  a  year.  May  and  June  are  the  highest  leads  generation  months,  followed 
by  November  and  January.  These  increased  lead  levels  are  consistent  with 
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Figure  9.  NRD  102  Leads  vs.  Dollars 
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Figure  10.  NRD  313  Leads  vs.  Dollars 
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Figure  11.  NRD  547  Leads  vs.  Dollars 
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Figure  12.  NRD  836  Leads  vs.  Dollars 
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potential  “job  seekers”  due  to  graduation  from  high  school  and  college.  It  is 
during  these  months  that  students  become  more  serious  about  their  future  work 
status  and  explore  their  alternatives.  Interestingly,  during  these  peak  lead 
months,  expenditures  consistently  decrease,  as  previously  illustrated  in  Figures 
4  through  8.  It  is  assumed  that  leads  will  occur  regardless  of  advertising  during 
these  times  of  the  year.  As  such,  peak  advertising  expenditures  coincide  with 
times  of  low  lead  rates  to  improve  awareness  and  promote  the  Navy  to  those 
potential  job  seekers.  Figure  13  clearly  illustrates  these  lead  patterns  from  FY 
1993  through  FY  1995. 

These  findings  also  validate  the  need  for  more  detailed  data.  Not  only 
should  the  data  already  discussed  be  collected  in  a  more  detailed  and 
consistent  manner,  but  other  extraneous  factors  which  influence  leads  should 
also  be  collected.  Many  of  these  factors,  however,  are  not  readily  measurable 
(i.e.,  family  influence,  recruiter  effectiveness,  personal  objectives,  etc.),  but  the 
more  detailed  the  data,  the  more  accurate  the  evaluation  of  advertising's 
effectiveness  will  be. 

The  unemployment  rate  is  one  external  factor  often  cited  as  being 
influential  in  an  individual's  determination  to  join  the  military.®  Contrary  to  the 
findings  of  past  studies,  when  examined  at  the  district  level,  correlations 
between  the  number  of  leads  generated  and  the  unemployment  rate  showed  no 


®  For  further  explanation  of  the  effect  of  unemployment  rate  on  potential  recruits  see  Hanssens 
and  Levien,  1983;  Warner,  1990;  or  Goldberg,  1979. 
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Figure  13.  National  Advertising  Leads  Generated  FY 1993  through  FY 1995 


53 


significant  association.  Findings  were  as  follows: 


Correlation 

NRD  102 

.01351 

NRD  313 

-.09905 

NRD  547 

.32276 

NRD  836 

-.04702 

These  findings  do  not  seem  to  support  the  usual  assumption  that  as  the 
unemployment  rate  rises,  so  do  the  number  of  leads  generated.  Subsequent 
analysis  using  additional  demographic  data  with  the  media-specific  leads  is 
necessary  to  further  investigate  this  hypothesis. 

The  New  Recruit  Survey  data  depicted  the  potential  effectiveness  of  the 
various  advertising  mediums  for  promoting  awareness.  Analysis  of  respondents' 
answers  to  the  advertising  awareness  questions  showed  significant  increases  in 
advertising's  effectiveness  in  November  1994,  following  the  doubling  of  the 
Navy's  advertising  budget  and  the  release  of  several  new  advertisement 
campaigns.  However,  following  this,  all  measures  remained  relatively  constant, 
with  a  minor  increase  in  awareness  by  the  August  1995  recruits. 

The  NRS  data  indicates  the  most  visible  of  the  advertising  mediums  is 
television.  As  seen  in  Figure  14,  when  asked  if  they  were  aware  of  advertising, 
over  70%  of  the  new  recruits  indicated  they  had  seen  Navy  television  ads. 
Magazines  followed  closely  with  an  approximate  60%  exposure  rate  for 
magazine  insertions. 
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Percent  aware  of  various  advertising  mediums 


Figure  14.  Navy  Advertising  Awareness 


55 


Newspaper  Magazine  Mail  Recruiter's  call  School/job  fair 


Analysis  of  advertising's  influence  on  the  decision  to  join  the  Navy 
revealed  that  the  recruiter's  personal  contact  was  by  far  the  most  influential.  A 
recruiter's  effect  on  an  individual  is  difficult  to  measure,  and  the  NRS  is  one  of 
the  only  sources  of  data  to  this  effect.  Further  analysis  at  the  district  level 
detailing  the  characteristics  of  the  influence  of  recruiters  may  expand  on  the 
recruiter's  role  in  promoting  awareness. 

Figure  15  presents  the  percentages  of  new  recruits  who  state  that 
different  advertising  mediums  affected  their  decision  to  join  the  Navy.  National 
advertising  mediums  consistently  create  influence  among  the  target  audience. 
Definitive  differences  exist  between  the  number  who  state  that  in  general, 
various  advertising  mediums  are  influential,  and  the  number  which  state  that 
advertising  had  "much"  or  "very  much"  influence.  National  advertising  mediums 
show  the  greatest  contrast;  recruiter's  phone  calls  remain  consistent. 

When  comparing  responses  before  and  after  the  budget  increase,  there 
does  not  appear  to  be  any  significant  change  in  the  influence  levels  of 
advertising.  The  level  of  influence  obtained  by  each  advertising  medium  has 
remained  relatively  constant.  The  fact  that  the  source  of  this  data  is  actual 
recruits,  makes  the  NRS  a  valuable  source  of  advertising  data.  As  such,  this 
lack  of  change  in  attitudes  since  the  budget  increase  is  cause  for  concern. 
Further  analysis  using  the  demographic  details  (i.e.,  hometown,  age,  education, 
job  experience,  etc.)  provided  with  the  NRS  data  might  reveal  reasons  for  the 
absence  of  any  consequential  change  in  the  target  audience  disposition. 
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Review  of  the  propensities  to  enlist ,  as  compiled  by  the  YATS,  reveal  the 
same  consistency  in  attitudes  among  the  target  audience.  In  1993,  16-21  year 
old  males  were  reported  to  have  a  9.8  percent  tendency  to  join  the  Navy.  In 
1994,  this  figure  dropped  to  8.6  percent,  and  rose  ini  995  to  9.7  percent. 

Though  the  figures  are  low  in  1995,  the  Navy  and  Army  were  the  only  services  to 
increase  their  propensities  by  over  a  full  percentage  point.  The  Air  Force  dipped 
.1  percent  and  Marine  Corps  rose  by  only  .2  percent.  The  lack  of  any  real 
change  in  the  attitude  of  the  target  audience  brings  into  question  whether 
increases  in  the  advertising  budget  results  in  change,  or  if  additional  funds  could 
be  better  used  elsewhere. 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  CONCLUSIONS 

The  primary  purpose  of  this  study  was  to  determine  if  recent  increases  in 
the  Navy  national  advertising  budget  resulted  in  an  increase  in  national 
awareness,  as  measured  by  the  number  of  leads  generated.  Due  to  data 
limitations,  this  study  provides  only  a  preliminary  analysis.  The  principle 
contributions  of  this  report  are  to  document  the  data  required  to  conduct  a  true 
cost-effectiveness  analysis  of  the  Navy's  advertising  expenditures,  and  to 
provide  a  preliminary  analysis  of  the  issue. 

Correlations  between  leads  generated  and  dollars  expended  reveal  the 
possible  existence  of  a  three-  or  four-month  lag  effect.  Shifting  the  leads  data 
four  months  behind  expenditures  results  in  a  positive  increase  in  the  relationship 
between  the  leads  and  dollars. 

The  study  verifies  the  association  between  impressions  and  levels  of 
spending.  Correlations  by  month,  media  type  and  NRD  revealcorrelations 
greater  than  .9,  as  expected.  However,  the  relationship  between  impressions 
and  leads  generated  cannot  accurately  be  determined  without  leads  specified  by 
media  type. 

Successful  execution  of  Navy  advertising  has  a  definite  effect  on  national 
awareness.  New  Recruit  Survey  advertising  awareness  questions  support  the 
effect;  recruits  answered  positively  to  both  being  exposed  to  various  mediums  of 
advertising,  and  being  "decidedly  influenced"  by  that  advertising.  No  evidence 
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exists,  however,  to  show  that  the  increase  in  advertising  expenditures  increases 
awareness  or  changes  attitudes  of  the  target  audience.  Responses  on  the  New 
Recruit  Survey  remained  relatively  constant  both  before  and  after  budget 
increases. 

The  NRS  reveals  that  the  target  audience  is  exposed  to  television 
advertisement  more  than  any  other  medium,  followed  by  magazine  insertions. 
These  mediums  may  be  the  most  cost  effective  means  of  reaching  a  large 
audience;  however,  further  regression  analysis  with  leads  by  media  type  is 
required  to  validate  this  conclusion. 

The  usual  assumption  that  an  increase  in  the  unemployment  rate 
increases  the  number  of  leads  is  not  supported  by  this  study.  The  correlations 
reveal  no  relationship  between  unemployment  rates  and  leads  generated. 

Again,  further  analysis  of  this  data  is  required  to  explain  this  relationship. 

B.  RECOMMENDATIONS 

In  order  to  accurately  determine  that  portion  of  the  advertising  budget 
which  contributes  to  production  overhead  and  advertising  execution,  the  B&E 
must  first  be  standardized.  The  difficulty  in  interpreting  this  budget  document 
only  contributes  to  the  potential  for  inaccurately  evaluating  expenditures.  The 
B&E  should  be  organized  consistently  each  year.  Expenditures  should  be 
organized  by  program  and  media,  in  a  consistent  manner  each  year.  If  possible, 
expenditures  should  be  maintained  by  month  to  facilitate  comparison  with  other 
expenditure  reports,  impressions  figures  and  leads  data.  Dollar  figures  must  be 
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kept  current,  with  fixed  costs  of  production  separated  from  the  variable  costs  of 
execution  and  administration.  Actual  purchase  or  contract  dates  should  be 
annotated  along  with  the  expected  view  dates  to  assist  in  determining  budget 
allocation.  Because  the  B&E  is  the  only  source  of  production  expenditures, 
standardization  of  the  report  would  facilitate  easier  compilation  of  production 
spending  each  year. 

The  NALTS  and  RIDS  both  provide  monthly  data  by  NRD;  however,  the 
two  databases  are  maintained  independent  of  each  other  by  two  different 
departments  within  CNRC.  NALTS  may  consider  adopting  the  format  used  by 
the  RIDS  as  it  enables  easy  data  manipulation  by  month  and  NRD.  Information 
from  the  NALTS,  currently  maintained  on  microfiche,  should  also  be  maintained 
in  a  computerized  database. 

In  order  to  conduct  a  more  thorough  analysis  of  the  relationship  between 
changes  in  the  Navy  advertising  budget  and  the  number  of  leads  generated,  the 
leads  data  must  be  maintained  by  media  type,  month  and  NRD.  The  use  of 
aggregated  television,  radio  and  magazine  leads  does  not  facilitate  an  accurate 
analysis.  Direct  mail  leads  are  also  essential  in  this  analysis.  Each  media  type 
has  a  different  effect  on  the  target  audience.  A  regression  analysis  using  media- 
specific  data,  by  month  and  NRD,  would  enable  a  determination  of  the  impact  of 
advertising  expenditures  on  leads  generation. 

One  source  of  important  demographic  data  are  the  New  Recruit  Surveys. 
Because  individuals  answering  these  surveys  have  already  accessed  into  the 
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Navy,  an  analysis  of  their  demographic  characteristics  might  prove  useful  in 
determining  better  allocation  of  advertising  resources. 

The  lack  of  association  between  leads  generated  and  dollars  expended 
begs  the  question,  how  does  CNRC  determine  the  mix  of  media  used?  Without 
leads  data  specified  by  media  type,  it  is  impossible  to  accurately  analyze  the 
impact  of  various  budget  expenditures  on  leads  generated  or  awareness.  Thus, 
CNRC’s  budget  decisions  are  complicated,  not  necessarily  optimal,  and  difficult 
to  justify. 
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